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Benchmarking has traditionally been viewed as a way to compare data only; however, its utilisation as 
a more investigative, research-informed process to add rigor to decision-making processes at the 
institutional level is gaining momentum in the higher education sector. Indeed, with recent changes in 
the Australian quality environment from the Australian Quality Assurance Agency (AQUA) to the 
Tertiary Education, Quality and Standards Agency (TEQSA), benchmarking has become a pragmatic 
tool in comparing themes and discipline standards across the sector. The paucity in benchmarking 
literature and institutional research is a key driver for universities to build capacity to use 
benchmarking both as a practical tool and as a research process to inform institutional decision¬ 
making. The University of Tasmania (UTAS) has been involved in seven benchmarking projects in 
the period from 2009 to 2012. This article will provide a brief overview of each benchmarking 
exercise and the lessons learnt along the way. It also outlines how the university has built capacity to 
inform strategic decision-making by using different types of benchmarking. 

Keywords: benchmarking, standards, institutional research, capacity building, collaboration 


Application of Institutional Research and Benchmarking in Higher Education 

The term ‘benchmark’ originated from surveying to denote a mark on a survey peg; 
however, in recent years it has acquired a more general meaning as a reference or criterion 
against which something can be measured (Jackson, 2001). Benchmarking as a process is 
both complex and comprehensive in terms of what type and what purpose it should be used 
for. Moreover, developing measures to evaluate multifaceted phenomena is also a complex 
task (Coates, 2010). There are many types of benchmarking, including; implicit/explicit; 
independent/collaborative; intemal/extemal; vertical/horizontal; inputs, process or outputs 
focused; quantitative/qualitative; and self-referencing against standards or expectations 
(Jackson, 2001). 

In the past, benchmarking has predominantly been used as a managerial, pragmatic 
tool to improve practice in the private sector in areas such as industry, manufacturing, retail 
and services (Moriarty & Smallman, 2009). The application of benchmarking in higher 
education (HE) is a relatively new phenomenon, first undertaken in the United States (US) in 
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the early 1990s in defining benchmarks and initiating benchmarking surveys (Epper, 1999). 
This was closely followed by the introduction of HE benchmarking activities in the United 
Kingdom (UK) in the early to mid-1990s (Yorke, 2004). More recently, however, 
benchmarking has been associated with league tables when comparing institutional 
performance (Burquel & van Vught, 2010). 

A key driver for benchmarking in the HE sector has been the establishment of quality 
assurance agencies in the UK, Europe and Australia to compare external reference points. 
Benchmarking from this perspective is apropos the comparison of standards. Subject 
benchmarking was introduced into the UK HE sector in 2000, when the Quality Assurance 
Agency (QAA) asked audit teams to focus on external reference points (i.e. subject 
benchmark statements) (Pidcock, 2006). Interestingly, Pidcock (2006) reported that close 
knowledge of benchmarking was not widespread among academics and anecdotal evidence 
suggests that this is still the case in Australia. In Europe, the key drivers have been the 
Bologna process and the Lisbon agenda, which have seen benchmarking exercises take 
standards a step further with performance targets set by institutions. Findings from a 
European benchmarking program with the European Centre for Strategic Management of 
Universities (ESMU; Swahn, 2004) identified the strengths of benchmarking to include not 
only a numerical comparison of data but also the ability to focus on the underlying processes 
of universities. 


Australian Universities Quality Agency 

Benchmarking first came to the fore in Australia when Australian Universities Quality 
Agency (AUQA) auditors, after their Cycle 1 audits, recommended that universities use 
benchmarking as a quality improvement strategy. AUQA then stated in its second Cycle 
(2008-2013) that it would expect HE institutions to demonstrate ‘How are standards 
compared nationally? How are they compared internationally? What explicit benchmarking 
has there been to compare standards?’ (AUQA, 2009). Stella and Woodhouse (2007, p. 18) 
also identified benchmarking as a quality improvement strategy and as ‘a systematic means 
of obtaining and specifying comparisons and learning from them’. A desktop review of Cycle 
2 AUQA audit reports found that Australian universities were at one of three stages of 
benchmarking development: 

• Stage 1—Early implementation: Universities identified at this stage were asked to 
urgently consider the development and implementation of a benchmarking framework, 
processes and partnerships as part of their Quality System. 

• Stage 2—Further refinement and alignment: Universities at this stage had developed 
and begun to implement benchmarking processes and partnerships, but further 
refinement and alignment with other universities was required. 

• Stage 3—Establishment: Universities at this stage had established benchmarking 
frameworks, processes and partnerships across the sector and made extensive use of 
external reference points and benchmarking (Booth, 2011). 

This desktop review also identified the types of benchmarking proposed by AUQA 
auditors: 

• benchmarking data (Course Experience Questionnaire [CEQ], Graduate Destination 
Survey [GDS], International Student Barometer [ISB], student load, research 
performance, international services, finance, equity etc.) 
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• standards-based benchmarking 

• sector benchmarking 

• whole-of-institution benchmarking 

• discipline-specific benchmarking (Booth, 2011). 

With the transition from AUQA to the Tertiary Education Quality and Standards 
Agency (TEQSA), benchmarking has become a useful methodological tool for comparing 
standards. However, the paucity of benchmarking literature and institutional research must 
now drive universities’ capacity to use benchmarking, both as a pragmatic tool and as a 
research process to inform institutional decision-making. The period from 2009 to 2012 has 
seen the University of Tasmania (UTAS) involved in seven benchmarking activities. A novel 
type of benchmarking recently introduced has been institutional-wide research benchmarking, 
of which UTAS has undertaken three such exercises and is currently involved in an 
international benchmarking project with the Higher Education Academy (HEA) on 
promotions policies. 

A brief overview of each UTAS benchmarking exercise and the key lessons learnt are 
provided below. 

Activity 1: Sector Benchmarking 

The initial benchmarking activity that UTAS undertook was a pilot project that 
investigated academic transition support for first-year undergraduate students in conjunction 
with the University of Wollongong (UOW) in 2009. This project was a direct result of 
recommendations made by AUQA auditors in their Cycle 1 audits, which asked both 
universities to undertake sector benchmarking in order to compare standards. Sector 
benchmarking occurs when benchmarking partners in the same sector make comparisons 
either as a whole organisation or an aspect of the organisation (Stella & Woodhouse, 2007). 
The aims of this pilot benchmarking project in academic transition support were to: 

• develop knowledge and experience in the benchmarking process 

• compare processes and practices 

• identify areas for improvement and areas of best practice. 

The benchmarking methodology was based on the e-leaming framework from the 
Australian Council on Open, Distance and e-Learning (ACODE, 2007). Quality teams from 
both universities used this framework as a basis to develop a unique framework on academic 
transition support. The academic transition support framework comprised ten performance 
indicators, good practice statements and performance measures (with ratings 1-5). A self¬ 
review process was undertaken at each institution, followed by a peer-review workshop. 

Table 1 provides a sample of the UTAS self-review results. 
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Table 1 

Sample ofUTAS Self-Review Results on Academic Transition Support 


Performance indicator 

Aligned plans and policies are in place and implemented 
Planning and delivery of programs/activities are coordinated 


Performance measure (rated 1—5) 

Rated 3 (Moderate) 

(a) Planning: Rated 4 (Considerable) 

(b) Delivery: Rated 4 (Considerable) 


The UTAS Academic Transition Support Final Report proposed eleven recommendations and 
provided a number of key findings at the institutional level: 

• The project identified areas of good practice at each institution and demonstrated that 
UTAS was comparable in performance standards with UOW on academic support. 

• UTAS rated well in comparison to the Australasian Survey of Student Engagement 
(AUSSE) survey (Australian Council for Educational Research, 2008). 

• Both universities were challenged in supporting equity groups. 

• The project highlighted that greater resourcing for student support services and greater 
support and induction for sessional staff in first-year coordination roles was required 
and that student evaluation data was not being utilised to its full potential. 

• The pilot benchmarking project increased UTAS’ understanding of benchmarking and 
highlighted areas where there was a paucity of evidence to support ratings. 

Both universities shared their final reports and identified collaborative actions from 
the benchmarking process, including: 

• defining a role for First-Year Coordinators 

• developing a first-year transition framework 

• developing administrative processes for at-risk students 

• comparing of library programs 

• professional development on first-year transition 

• comparing International Services. 

Each of the above actions involved staff from the two universities collaborating and 
sharing their processes and practices in academic transition support. 

Activity 2: Process and Academic Standards Benchmarking 

The second benchmarking project (January 2010-November 2010) was based on 
assessment policies and processes and involved both UTAS and UOW, thereby building on 
relationships developed from the first benchmarking exercise. In addition, Deakin University 
was brought on as a benchmarking partner. The project’s main reference point was the 
Australian Learning and Teaching Council (ALTC) Teaching Quality Indicators Project 
(TQIP; Davies, 2009). Three of the four TQIP indicators were adapted as performance 
indicators: 

• Assessment purposes, processes and expected standards of performance are clearly 
communicated and supported by timely advice and feedback to students. 
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• Assessment practices and processes are fair, reliable and valid and produce marks and 
grades that represent the standards achieved by students. 

• Assessment policies and procedures are developed, implemented, reviewed and 
improved in accord with policy principles. 

Due to the experience gained from the initial benchmarking project, the ratings were 
changed to encourage conversation between staff instead of focusing on ratings only (i.e. 1— 
5). For example, Yes', Yes, but; No, but; No. Another lesson leamt was that evidence needed 
to be provided to support the ratings. 

Each benchmarking partner institution applied slightly different methodologies due to 
their contextual differences, size and resources available. The chief research instrument used 
by UTAS was an online survey sent, which was completed by 336 staff and was based on the 
assessment benchmarking framework, but rephrased into questions to promote more 
meaningful responses. Seven postgraduate students and one Research Fellow analysed the 
staff survey data, which was then utilised to inform the development of scoping papers for 
each faculty. To validate the analysis of the data and include additional contextual 
information, an interview process was subsequently undertaken within each faculty. A self¬ 
review process with the Faculty Associate Deans (Fearning and Teaching) also validated the 
process and included agreed ratings at both the faculty and institutional level. The assessment 
benchmarking project also included a benchmarking of assessment at the Academic Senate 
level to ensure standards in assessment were compared. 

The findings for UTAS in the assessment benchmarking project are provided below: 

• UTAS demonstrated that the implementation of criterion-referenced assessment 
(CRA) was critical in aligning learning outcomes and generic graduate attributes, and 
assisted in demonstrating student achievement standards. 

• Associate Deans (Fearning and Teaching) lacked time and resources to drive quality 
improvement projects. 

• Four factors affected the timely return of feedback: working in an online environment, 
moderation, large student cohorts and multi-campus sites. 

As a result of UTAS’ participation in the assessment benchmarking project, 27 
recommendations were proposed and an Assessment Benchmarking Working Party was 
established to ensure the recommendations were turned into outcomes. The latter is an 
essential component of all benchmarking activities in order to justify the activity and enhance 
practices that is discussed further below. The benchmarking project also identified areas for 
sharing and collaboration between the three universities, including: 

• online subject outline templates 

• best practice case studies on group work assessment 

• online training manual for assessment 

• UTAS Graduate Attributes Project (Faculty of Business) 

• Deakin Faculty Fearning and Teaching Funding Model 

• postgraduate research. 

Postgraduate research was identified as an area for learning by the Chairs of Senate 
from the three universities during the Academic Senate peer-review workshop on assessment. 
While UTAS had already undertaken a self-review of higher degree research (HDR), the 
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indicators developed from this review informed the development of a HDR benchmarking 
framework—Activity 4 below (Booth & Frappell, 2011). 

Activity 3: Information Benchmarking 

An institutional information benchmarking exercise was undertaken (November 
2010-January 2011) to inform the development of a UTAS Benchmarking Policy. An 
analysis of this benchmarking information identified five themes: 

• The distinction between benchmarks (for comparison purposes only) and 
benchmarking more as an investigative process for quality improvement purposes need 
to be clarified. 

• While the majority of organisational units undertook regular benchmarking activity, 
these were used predominantly for comparative purposes rather than planned 
benchmarking projects based on process and outcomes. 

• The type and level of benchmarking activity varied across the institution (e.g. 
horizontal, outcome, performance and public information benchmarking). 

• Administrative and support services were strong in functional benchmarking. 

• Benchmarking did not include membership on committees, staff born overseas, 
visiting scholars, informal conversations, exchange programs, membership on editorial 
boards or consortiums or external networking groups. 

Activity 4: Process and Outcomes-based Benchmarking 

A higher degree research (HDR) benchmarking project (March 2011-March 2012) 
involving UTAS, UOW and Deakin was considered mutually beneficial for developing a 
shared conversation around HDR processes and outcomes. The performance indicators were 
based on a self-review of HDR processes and outcomes (Booth & Frappell, 2011). The 
performance indicators included: policy and governance; selection and admission processes; 
student learning outcomes; supervision; examination processes; academic support, including 
mentoring and career placement; non-academic support and resourcing; monitoring of student 
performance and feedback data. Data were also compared on student load, postgraduate 
coursework, HDR EFTSL, % HDR and staff FTE. 

Examples of good practice identified across the three universities included: 
comprehensive policy suites, multiple modes for communication of policies and procedures 
to supervisors and staff, orientation and induction processes, robust annual review processes, 
and rigorous examination processes. The areas identified for improvement included: more 
systematic use of data for quality improvement purposes, monitoring English language entry 
requirements, clearer articulation of learning outcomes aligned to the Australian 
Qualifications Framework (AQF), more systematic English language support, HDR student 
career development, and clearer articulation of the role of supervisors in provision of 
academic support. 

Areas identified for further collaboration included: the development of programs for 
student and supervisor workshops, Graduate Certificate in Research, sharing Head 
Postgraduate Studies (UOW) and Graduate Research Coordinator position descriptions 
(UTAS), the development of a fast-track supervision program (Deakin), sharing HDR 
Newsletter (UTAS), integrated PhD (UOW), and sharing statement of HDR learning 
outcomes/graduate attributes (UTAS). The three Deans of Graduate Research also put 
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forward a joint application to the Brazilian Consortium (CAPES) to build international HDR 
collaborations. 

Activity 5: Standards Benchmarking 

In Semester 1, 2012, UTAS participated in Phase 2 of the national Teaching 
Standards Framework (TSF) Project (Sachs & Kosman, 2012) led by Macquarie University. 
The TSF Project was initiated to facilitate the development of (a) a comprehensive teaching 
framework that could be used for both regulatory and developmental purposes for the 
Australian HE sector, and (b) a robust online version of the TSF by which institutions could 
conduct a comprehensive survey of their processes for ensuring teaching quality, and to 
report accordingly. Twelve institutions, including UTAS, participated in testing the online 
version of the TSF. Each participating university undertook a self-review of each of the 
teaching standards, which included testing the validity of the TSF. UTAS completed online 
TSF reports at course, faculty and institutional level. The course chosen to participate in the 
project was the Bachelor of Education (Primary), which was assessed by evaluating units in 
the course. The faculty chosen was the Faculty of Arts. Given the short time frame allocated 
to complete the evaluation, faculty level was assessed by evaluating Faculty of Arts first-year 
units only. 

Due to UTAS’ previous involvement in institutional benchmarking exercises, it was 
identified early on that the TSF performance indicators were aimed predominantly at the 
institutional level and acknowledged that this would be difficult for staff responding at the 
unit/course level. Subsequent to discussions with the relevant Associate Deans (F&T), it was 
decided to develop an online survey that focused only on the relevant criteria to staff as 
Course and Unit Coordinators. Similarly to the assessment benchmarking project, the TSF 
performance indicators were each framed as a question, rather than a statement. This 
provided more guidance and promoted discussion regarding evidence sources. For example, 
rather than stating ‘Stakeholders provide feedback into, and on, teaching plans’, this was 
rephrased as ‘Do you incorporate feedback from stakeholders into your teaching plans? If yes, 
what are your sources of this feedback?’ 

Data obtained from the course and faculty surveys were then analysed and utilised by 
the UTAS TSF Project Officer to complete the online TSF course and faculty reports on 
behalf of the faculties and also at the institutional level. A key lesson learnt from this process 
was the benefits that are associated with the development of an online software tool for 
benchmarking purposes (e.g. a clear and consistent system for evaluation and reporting 
against the performance indicators). UTAS has since developed its own software tool for 
benchmarking standards both institutionally and across the HE sector. 

Activity 6: Discipline-Specific Benchmarking 

As part of an 11-university project led by the University of Western Sydney, UTAS, 
UOW and Deakin recently participated in an inter-university peer review and moderation 
project (Krause et al., 2010). The three universities participated in the project in Semester 1, 
2012 due to their strong benchmarking relationships. The disciplines benchmarked were 
Economics, Journalism, History and Nursing. The discipline specialists from each partner 
university compared: 

• the learning outcomes, assessment criteria, and assessment tasks in common units of 
study 
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• the reliability of marking in these units. 

While this project is ongoing, it has identified unit-level moderation processes for 
benchmarking across the disciplines. 

Activity 7: International Institutional Research Benchmarking 

Building on a 2009 Higher Education Academy (HEA) report on reward and 
recognition and internationally recognised work at UOW on promotions criteria and peer 
review of teaching-related activities, this recently commenced international benchmarking 
project (May 2012-May 2013) aims to produce resources to guide and improve academic 
promotion policy and practice to reflect the recognition of teaching as core to the assurance 
of standards in HE (Cashmore et al., 2012). The project, funded by the HEA, involves two 
British universities (University of Leicester and University of Newcastle upon Tyne) and two 
Australian universities (UTAS and UOW) and includes the development of a theoretical 
benchmarking framework for academic promotion that can be applied across the HE sector. 

A core research project team is leading the benchmarking project and an international 
advisory group and an external evaluator have been appointed to oversee the peer-review 
process and evaluation of the project. 

Key Lessons Learnt 

Benchmarking must be Translated into Institutional Research 

The key to ensuring that benchmarking becomes an institutional research process is 
identifying the issue(s) that benchmarking is trying to resolve (Longden & Yorke, 2009). For 
benchmarking to inform institutional decision-making it has to be more than just the 
comparison of benchmark data. As identified by Longden and Yorke (2009, p. 67), 

‘Volkwein has provided a useful matrix to explore the nature and form of data within the 
context of higher education’. Figure 1 depicts Volkwein’s matrix as adapted to suit the UK 
context (Longden & Yorke, 2009). The matrix has been further adapted (in italics) to 
correspond to benchmarking (Booth, 2012). 

Cell 1 has an administrative/managerial, internal, formative focus that is centred on 
how a university manages its raw data. In applying this matrix to benchmarking, the 
university is reflecting on ‘Is this what we look like? How do we compare to others? How 
can we improve?’ The focus is on the comparison of raw data as benchmarks. The 
information benchmarking activity to inform the development of a UTAS Benchmarking 
Policy described above found that this was the most common form of benchmarking for 
UTAS staff and that it is predominantly used for comparative purposes. 
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Direction of Focus 


Internal External 

Formative Summative 


Administrative 

Staff/ 

Managers 

1. Information 

- Benchmarking of Data only 

2. Presentation 

- Sector Benchmarking 

- Ranking 

Faculty 

3. Research for 

4. Educational 


Improvement 

Research 

Senior 

- Standards Benchmarking 

- Standards Benchmarking 

Executives 

- Projects informed by 

- Projects informed by 


research 

research and validated by 



external reference groups 


Figure 1. Focus and role of institutional research in benchmarking. 

Source: Longden and Yorke, 2009 (adapted from Volkwein, 1999, p. 17); Booth (2012). 

Cell 2 has an administrative/managerial, external focus that is centred on 
benchmarking for the summative purposes of institutional funding and government reporting 
(e.g., ERA and MyUniversity data). The benchmarking of data in this cell is to support the 
acquisition of other resources for the university. Benchmarking data is sensitive and open to 
alternative interpretations and is used by the university and others for ranking purposes. Thus, 
the focus is on sector and functional benchmarking and ranking. 

Cell 3 has an academic, internal focus that is both internally and externally generated 
and leads to quality improvement at the operational level. The types of benchmarking 
projects pertinent to this aspect are centred on solving a problem or theme, and provide 
evidence on which both individuals and organisational units can make decisions regarding 
their professional practice. Presently, the focus of these benchmarking projects is evaluation 
of performance against a set of standards. 

Cell 4 has an academic, external focus that deals with issues beyond the internal 
interests of the university. Longden and Yorke (2009, p. 69) argue that ‘research conducted 
with an external focus can come close to fulfilling the methodological expectations of 
educational research’. This cell identifies standards benchmarking and institutional research 
benchmarking, for instance, the HEA promotions benchmarking project (Cashmore et al., 
2012). The external focus is also represented by external reference groups and external 
reporting requirements. 
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Benchmarking is Justified by Driving Operational Outcomes That Improve Processes 

For benchmarking to be successfully implemented in universities it has to become the 
‘way things get done at universities’ (Hossler, Kuh, & Olsen, 2001, p. 212). Benchmarking 
has to be clearly articulated into an institutional process for organisational improvement with 
appropriate resourcing to ensure that staff are provided with support. UTAS has recently 
established a strategic institutional unit—Student Evaluation, Review and Reporting Unit 
(SERRU)—to ensure that benchmarking is a key strategy for institutional organisational 
improvement and change, and to support the university in building capacity to undertake 
institutional research to inform strategic decision-making. Benchmarking initiatives must also 
be reflected in strategic and faculty plans in order to drive quality improvement. 

A Rigorous Methodological and Theoretical Approach is Essential 

Literature on how to build a theory of benchmarking is limited (Liang, as cited in 
Moriarty & Smallman, 2009, p. 487). The benchmarking format used across the UTAS 
benchmarking projects discussed in this article is derived from the ACODE framework 
(2007) and literature on benchmarking (Jackson, 2001; McKinnon, Walker, & Davis, 2000). 
There are also nine essential phases that must be considered in order to undertake a sound and 
robust benchmarking project: (1) determination of which areas to benchmark, (2) 
identification of benchmarking partners, (3) determination of the types and level of 
benchmarking, (4) preparation of benchmarking documentation/framework, (5) design of 
benchmarking process, (6) implementation of the benchmarking process, (7) review of results, 
(8) communication/reporting of results and recommendations and (9) implementation of 
improvement strategies (Booth, Melano, Sainsbury, & Woodley, 2011). An online 
benchmarking template is also considered useful for ease of access, increased efficiency, 
consistent reporting and secure storage of evidence. 

In addition to a rigorous methodological approach is the development of a functional 
and appropriate theoretical framework. Moriarty and Smallman (2009) argue that a lack of a 
theoretical framework in benchmarking distinguishes effective from ineffective efforts. The 
UK benchmarking project (Cashmore et al., 2012) on promotions has a theoretical framework 
with performance indicators derived from literature in the field. 

Benchmarking as a Community of Practice 

For benchmarking exercises to be truly successful there has to be a bottom-up 
empowerment (Ellis & Moore, 2006) where there is sharing and articulation of expertise. 
Sciulli, Smith and Ross (2009) argue that ‘Collaboration and openness amongst participants 
are key ingredients for a successful benchmarking exercise’ (Sciulli, Smith, & Ross, 2009, p. 
13). Benchmarking becomes a shared conversation and can be viewed as a form of peer 
development (Leppisaari et al., 2011). It has also been seen as a form of learning (Ellis & 
Moore, 2006), the process thereby adhering to those universities that characterise themselves 
as learning universities (Pidcock, 2006). 

As a university, UTAS has learnt a great deal, not only about the complexities, but 
also the rewards of benchmarking that include: close collaborations, building of cross- 
institutional relationships, increased research outcomes, and improved operational processes. 
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